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Abstract 

We performed group theoretical investigation of symmetries of excitations in 
topological insulators Bi 2 Sb 3 , Bi 2 Te 3 , Bi 2 Se 3 and Sb 2 Te 3 , focusing on selection rules 
for optical processes. Electronic transitions of bulk states to bulk states, surface 
states to surface states and bulk states to surface states are studied over the entire 
Brillouin zone. A new technique is used to deal with transitions between surface 
states and bulk states. Time reversal symmtry is also included in the analysis. 
Our results show that only T, A and M points of the Brillouin zone would depend 
on light polarization for transitions between bulk states and surface states. As an 
example of application, electron spin polarizations of photoelectrons are calculated 
at T point. A general rule for the direct product between the representation at an 
arbitrary point in the Brillouin zone with itself is presented. 

I. INTRODUCTION 

Topological insulator (TI) exhibits a novel quantum state [1, 2, 3, 4, 5], with topo- 
logically protected conducting surface states existing within the bulk energy gap [6, 7, 8, 
9, 10, 11]. In addition to their fundamental interest in quantum mechanics, such as Ma- 
jorana fermions [12], the surface states are predicted to have applications in spintronics, 
quantum computation and others [13, 14, 15, 16]. The semiconducting V 2 VI 3 com- 
pounds Bi 2 Sb 3 , Bi 2 Se 3 , Bi 2 Te 3 and Sb 2 Te 3 , formerly known as promising thermoelectric 
materials with figure of merit ZT = 2.5 [17], are now identified as three dimensional topo- 
logical insulators. They crystallize in the layer structure (see Fig. la) of rhombohedral 
symmetry R3m (D^ d ) [18]. 

Compared with usual transport measurement techiques, optical measurements have 
the advantage that no direct contacts are required. Due to the presence of impurities, 
however, contribution from the surface states is masked by bulk carrier response in opti- 
cal experiments [19, 20, 21]. Surface sensitive tools, especially ARPES and STM, allow 
observations of genuine surface states. A full group theoretical study of selection rules is 
helpful for optical experiments because using different polarizations, contributions from 
the surface and the bulk might be separated. In this paper, we report optical selection 



rules over the entire Brillouin zone. Three kinds of electronic transitions are considered: 
bulk to bulk, surface to surface and bulk to surface. To apply group theory to transi- 
tions between bulk states and surface states, a smaller group consisting of all common 
elements from both groups is constructed. To our knowledge, this is the first time this 
method has been applied to bulk-surface physics problems. 

This paper is arranged as follows: In section II the geometry of these TI materials 
is reviewed, for both bulk structure and surface structure; Section III introduces three 
dimensional (3D) and two dimensional (2D) space groups. Also given is the method 
to construct a smaller group, in which the selection rules between bulk state to surface 
state is calculated; Symmetries of all electronic wave functions for bulk state and surface 
state over the entire Brillouin zone, as well as symmetries of the surface phonon modes, 
are given in section IV ; Section V gives detailed selection rules at different Brillouin 
zone points and section VI applies the analysis to electron spin polarization calculation 
of photoemission process at T point. Dissusions are given in section VII, where the use 
of P6mm space group is justified. 



II. CRYSTAL STRUCTURE AND BRILLOUIN ZONE 

In this section, Bi 2 Se 3 is used to illustrate the bulk and surface structures of V-VI 
compounds. Bi 2 Sb 3 , Bi 2 Te 3 and Sb 2 Te 3 have the same structure although the lattice 
parameters are different [18]. 

II. A Crystal structure and Brillouin zone of the bulk 

The crystal structure of Bi 2 Se 3 is given in Fig. la. The stacking of atoms in -z 
direction is in the order of: Se2(A) - Bi(B) - Sel(C) - Sel(A) - Bi(B) - Se2(C) - Bi(A) - 
Sel(B) - Sel(C) - Bi(A) - Se2(B) - Bi(C) - Sel(A) - Sel(B) - Bi(C) - Se2(A). Surface A, 
B, C sites are shown in Fig. lb. Two neighboring Bi layers are weakly bonded by Van der 
Waals interaction and are most likely to produce surfaces. The crystal structure belongs 
to space group R3m in Hermann-Mauguin notation and D^ d in Schoenflies notation 
(#166). The lattice parameters are a = 4.138 A and c = 28.64 A for Bi 2 Se 3 [18]. Fig. la 
also shows the Bravias lattice vectors: 

— —a —a c 

ai = ( !"'2vT3 ); 

— a —a c 

° 2 = ( 2'2~7I'3 ); 

— —a c 

This is the rhombohedral translational group R. The hexagonal unit cell contains nine 
formula units and the primitive cell contains one formula unit. Two Bi atoms and two Sel 
atoms occupy 2c Wyckoff positions and one Se2 atom occupies la Wyckoff position [18]. 
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The corresponding reciprocal lattice vectors are: 



p -2tt -2tt 2ti\ 

b i = — i—^Ti— ); 
a V3a c 

r ,2vr -2vr 2vr. 

6 2 = — .-7ir>— ; 

a V3a c 

r* . -47T 27T. 

V3a c 

The first Brillouin zone is shown in Fig. lc. High symmetry points on the surface are 
labelled. 



II. B Crystal structure and Brillouin zone of the surface 

Surfaces are most likely to form on the (111) plane, between the layers. There are 
three types of possible surface structures, labeled A, B and C in Fig. lb. B and C differ 
from A by a fractional translation (a/2, +a/2\/3, All three surface structures be- 

long to "wallpaper" group P6mm (see section III. III. B for detail). The lattice parameter 
is a = 4.138 A. Fig. lb also shows the Bravias lattice vectors of the surface structure: 



a'i = (a, 0,0); 
a', - (~ 2 — ,0). 
They form translational group P and can be decomposed using the bulk Bravais lattice: 

a'\ = a 2 — ai; 
(12 = a?, ~ ai ■ 

For all A, B and C surfaces, there is only one atom in one primitive cell. It occupies 
WyckofF position 1(a) [18]. The reciprocal lattice vectors are: 

b'i = -,-^-,0; 
a V3a 

J Air 
b' 2 = (0,-^,0). 
V3a 

The first Brillouin zone of the surface is shown in Fig. Id, with high symmetry points 
labeled T, M, K, high symmetry lines labeled A, S, Z and arbitrary point R. Brillouin 
zone labelings are adopted from reference [22]. The Brillouin zone of PQmm is only a 
part of the k z = cut of the Brillouin zone of R3m (Fig. Id). The zone boudary points 
of PQmm is still inside the first Brillouin zone of R3m. 
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Figure 1: (a): Crystal structure of Bi 2 Se 3 from [9]. (6): three different types of sur- 
faces. The Brillouin zone of the bulk and the surface are shown in (c) and (d). The 
Brillouin zone of the surface is only part of the k z = cut of the bulk Brillouin zone (the 
outter hexagon in d). The labeling of the Brillouin zone points follows reference [22]. 
A schematic band structure containing both bulk states and surface states is shown in 
(e) along the T-M direction. The crossing point of the two surface bands has linear 
dispersion and is called Dirac point. The details of the band symmetry labelling are in 
section IV. A. 
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III. SPACE GROUPS 



Group theory deals with symmetries. Its applications in physics problems are based 
on the matrix-element theorem: 



I 



il>Hct> dr # 



(1) 



only if Ti e ®D H ip, H, <j> are wave functions and operators; D^, D H , are 

the representations according to which tp, H, <p transform; (x) is the direct product and 
r"i is the identity representation. This theorem leads to well known selection rules in 
infrared absorption, Raman scattering as well as many others [23]. To avoid confusions, 
we use Di, di and Si to denote the zth representation of space group R3m, PQmm and 
P3ml, respectively. P3ml is a subgroup of both R3m and PQmm: 



R3m 



PQmm 



P3ml 



III. A Three dimensional space group 

The crystal structure of Bi 2 Se 3 belongs to space group R3m (D^ d , #166). The sym- 
metry operations are [24]: {E\0}, {Cf |0}, {C 6 4 |0}, {C x \0}, {CiC x \0}, {C 6 4 C,|0}, {I\0}, 
{/ClIO}, {/C||0}, {IC x \0}, {IC£C x \0}, {IC$C x \0}, as well as their products with trans- 
rational group R. All symmetry operations have their usual meanings [25]: E is the 
identity operation, C™ L is the rotation counterclockwise by 2ir ■ n/m, C x is the rotation 
along x axis by ir and / is the inversion. Symmetry elements are expressed in terms of 
generators for better presentation. Operations C\ are written down as C|*, which makes 
connections to two dimensional space groups more clear. Space group R3m is symmor- 
phic. The character table of irreducible representations at any point of the Brillouin 
zone can be easily obtained from their corresponding point groups [25]. When optical 
transitions connecting two bulk electronic states are considered, both electronic wave 
functions ip and 4> of equation (1) transform according to certain irreducible representa- 
tions of R3m. Direct product within R3m provides selection rules. 

III. B- Two dimensional space group 

Symmetries of surfaces can be described by two dimensional space groups, which 
are also called "wallpaper group", "plane symmetry group" or "plane crystallographic 
group". There are totally 17 wallpaper groups, catagorized into 4 crystal systems 
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(oblique, rectangular, square, hexagonal) and 5 Bravais classes (p-oblique, p-rectangular, 
c-rectangular, p-square, p-hexagonal), just as 230 3D space groups belong to 7 crystal 
systems and 14 Bravais classes. 

The structures of type A, B and C surfaces are the same (see Fig. 16). The Bravais 
lattice class is p-hexagonal and the crystal system is hexagonal [22]. Symmetry operations 
are [24]: {E\0}, {C 6 |0}, {Cf |0}, {Cf|0}, {C 6 4 |0}, {C||0},{/C x |0}, {/C 6 C x |0}, {IC$C x \0}, 
{ICqC x \0}, {ICqC x \0}, {ICqC x \0}, as well as their products with translational group P. 
This is PQmm wallpaper group (#17). P6mm is symmorphic and point group character 
tables are sufficient for all irreducible representations in the entire Brillouin zone. Both 
V> and <fi of equation (1) transform according to certain irreducible representations of 
PQmm. Direct products within P6mm give all the selection rules. We abandoned the 
barred notations for surface states and it should not be confusing. 

III. CZ Transition between 3D and 2D 

When optical transitions happen between a bulk electronic state and a surface elec- 
tronic state, the above calculations need to be modified. Let ip denote the bulk elec- 
tronic state that transforms as Di of R2m and <p denote the surface electronic state that 
transforms as dj of P6mm. The matrix-element theorem does not apply to Di and dj 
because they are representations of different groups. In order to apply group theory, 
a smaller group G' must be constructed, containing all common symmetry operations 
from both R2m and PQmm. In the smaller group G' , Di of R3m restricts to irreducible 
representation Si of G' and dj of P6mm restricts to irreducible representation Sj of 
G' . Matrix-element theorem can now be applied since Si and Sj are both irreducible 
representations of G'. 

The common symmetry operations of R2m and PQmm are: {-E|0}, {C||0}, {C||0}, 
{/CrlO}, {ICqC x \0}, {ICqC x \0}, and their products with translational group P. This 
is the wallpaper group .P3ml (#14) [22] and it belongs to hexagonal crystal system 
and p-hexagonal Bravais class. It is also symmorphic. The same translational group P 
as in P6mm leads to same Brillouin zone. However, they have different point groups: 
Cs v (3m) for P3ml and CQ V (6mm) for P6mm. This is important in later calculations. 
For example, S point and £' point in Fig. Id are equivalent in P6mm while in P3ml 
they are inequivalent. 

IV. ELECTRONIC WAVE FUNCTIONS and SURFACE PHONONS 

IV. A Symmetries of electronic wave functions 

In the tight binding approximation, the electronic wave functions are formed by 
atomic orbitals localized at individual atoms. The symmetry of the wave functions is 
the direct product of the permutation representation and the atomic orbital representa- 
tion. The permutation representation takes atomic positions as basis functions and its 
characters are the number of atoms unchanged or can be shifted back to itself through 
a lattice vector. The atomic orbital representation is simply the representation formed 
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Table 1: Electronic wave function symmetry in R3m. 
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Table 2: Electronic wave function symmetry in P6mm. 



by basis functions of atomic orbitals: (x 2 + y 2 + z 2 ) for s, (x, y, z) for p and so on. 

The electronic wave function symmetries for the bulk are given in Table 1. The five 
atoms in R3m occupy two Wyckoff positions 2(c) and 1(a). Wave functions are listed 
for atomic orbitals s, p, d, f, which include all atomic orbitals in Bi, Se, Sb and Te. 
Table 2 lists possible electronic wave function symmetries for the surface where atoms 
only occupy 1(a) Wyckoff position. Wave fucntions are also listed for atomic orbitals s, 
p, d, f. The surface states in Bi 2 Se 3 are formed by atomic orbitals p z [9] and they belong 
to representation T±, A 2 , E 2 , Mi, K\, Z 2 and R\ in the surface space group P6mm.The 
lowest conduction band and highest valence band of bulk electronic states are formed 
by atomic orbits P x ,P y [9]. Taking into account of electron spins, the electronic bands 
along the TM line in Fig. le should be labelled T7 and A3 © A4 for the surface states 
and T 4 _ , T5_ © Tq- and A3 © A4 for the bulk states. 

IV. B Symmetries of surface phonon modes 

For the sake of completeness, we list also the surface phonon structures of TIs (the 
phonon structure of the bulk Bi 2 Se 3 is well known [26]). With the assumption that the 
surface modes are strongly localized and symmetry classification of surface phonons can 
be carried out within P6mm space group. Ti and Tq phonon modes are Raman active 

in Cq v point group. The corresponding two dimensional Raman tensors are: ^ ) for 
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Table 3: Surface phonon modes of Bi 2 Se 3 at different Brillouin zone points, x, y row gives 
atomic displacements confined in the surface while z row gives atomic displacements out 
of the surface. 

V. OPTICAL SELECTION RULES 

In this section, all optical selection rules at different Brillouin points are tabulated. 
In each Brillouin zone point, the group of wave vector Go(k) and factor group G/Go(k) 
are listed. The tables for optical transition selection rules are obtained by performing 
direct products within R3m and P6mm, as well as direct products between R3m and 
P6mm. Restrictions from R3m and P6mm at different Brillouin zone points are given 
in Table 4. Only zone center representations are listed in the direct product because 
only those participate in single photon optical transitions. For bulk electronic bands, 
only states lies in the k z = plane can be optically excited to surface electronic states. 

V.A Direct products for space groups 

The direct product of two irreducible representations i}(**0( m ) (x) D(*^"')( m ') is in 
general reducible and the reduction coefficients (irkmir k'm'\ -kk"m") can be determined 
once the irreducible representaion D^* K ^ m ^ itself is obtained [27]. Rotational operations 
bring wave functions with wave vector k to wave functions with wave vectors {ip • k}. 
Some of these wave vectors are equivalent to k: {ipi ■ k} = k + Bh, where E>h is any 
reciprocal lattice vector. The inequivalent set of {<p ■ k} is defined as star of k and each 
inequivalent one of {ip ■ k} is called an arm of the star. Those symmetry operations 
{<p\t(<p)} that {<fi • k} = k + Bh defines a new group Go(k). It is called group of wave 
vector k and is a subgroup of G. Irreducible representations of space group G can be 
induced from irreducible representations of space group Go(k): D( fc )( m ) f Z)(* fc )( m ). 

Time reversal operation 6 is also a symmetry operation of the system. Taking into 
account the time reversal operator 9, a larger group H = G © 9G is obtained. H is the 
(second type) magnetic space group and G is the original space group of only spatial 
operations. The represenation of H can be induced from G and this problem was treated 
by Dimmock and Wheeler [28]. It can be shown [27] that if —k $ -kk, there is extra 
degeneracy that representations from *— k and -kk are forced to stick together. When 
—ke-kk, there are still possibilities that time reversal operation connects two irreducible 
representations, such as T 2 and r 3 of C^ v . Those representations must appear in pairs, 
unless magnetic field breaks the time reversal symmetry. Representations from its time 
reversal star *— k are labelled as D* , such as Ai and A* in table 4 are representations 
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Table 4: Restrictions from P3m and P6mm to P3ml. The * means representations 
from its time reversal star. K 2 and K 3 are also connected by time reversal symmetry, 
although they belong to the same star -kK. 



at *A and *-A of P3ml. 



V.B T point 

r = (0, 0). In R3m, G (k)= ({E\0}, {C 6 2 |0}, {C 4 |0}, {C x \0}, {CgC x |0}, {C 6 4 C,|0}, 
{/|0}, {IC$\0}, {/C 4 |0}, {IC x \0}, {ICIC X \0}, {IC$C x \0}) = D 3d , G/G (k)=({E\0}). In 
P6mm, G (k) = ({E\0}, {C 6 |0}, {Cf|0}, {Cf|0}, {C 4 |0}, {Cl\0},{IC x \0}, {IC%C x \0}, 
{IC$C x \0}, {ICiC x \0}, {IClC x \0}, {ICiC x \0}) = Cev, G/G (k) = ({E\0}). In P3ml, 
G (k) = ({E\0}, {Cf|0}, {C 4 |0}, {IC x \0}, {JCf C*|0}, {IC%C x \0}) = C 3v , G/G o (k)=({E\0}) 



V.C A point 

A = (V%/2,£/2). In R3m, G (k) = ({E\0}, {/C 6 2 C,|0}) = C a , G/G (k) = ({E\0}, 
{C 6 2 |0}, {C 4 |0}, {/|0}, {ICi\0}, {/C 4 |0}). In P6mm, G (k) = ({E\0}, {IC 3 C x \0}) = C s , 
G/G (k) = ({E\0}, {C e \0}, {C||0}, {C||0}, {C 4 |0}, {C||0}). In P3ml G (k) = ({E\0}, 
{IC%C x \0}) = C s , G/G (k) = ({E\0}, {Ci\0}, {C 4 |0}). 

As mentioned previously, — A^*A in P3ml, while — Ae*A in R3m and PQmm. This 
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Table 5: Optical selection rules at T point. 







R3m 


P6mm 






Ai A 2 


Ai A 2 


R3m 


Ai 
A 2 


rii^r^ r 2 r3r 4 r 6 
Ti ,r 3 r 4 r 5 


TiTs r 2 T3 
r 2 i"3 riT3 


P6mm 


Ai 
A 2 




rii^i^r^ r 2 r3rsr6 
rir 4 r 5 r 6 



Table 6: Optical selection rules at A point. 



has two effects: representations of Aj of R3m and P&mm must decompose into pairs of 
Aj and A* in P3m\ (see Table 4), where A* is irreducible representations of *— A; zone 
center representations are contained in Aj®A* instead of in Aj®Aj and Af(g>A*. This is 
also the case for Z point and R point. 



V.D £ point 

E = (£,0). In R3m, G (k) = ({E\0}, {C x \0}) = C 2 , G/G (k) = ({E\0}, {C||0}, 
{C 4 |0}, {I\0}, {/C||0}, {/C 4 |0}). InPGmm, G (k) = {{E\0}, {/CfC x |0}) = C„ G/G (k) 
= ({E\0}, {C 6 \0}, {C||0}, {Cl\0}, {C 4 |0}, {Cl\Q}). In P3ml, G (k) = ({E\0}) = C u 
G/G (k) = ({E\0}, {Cl\0}, {C£\0}, {/|0}, {ICi\0}, {K7 4 |0}). 



V.E M point 

M = (7r/a,7r/V3a). In R3m, G (k) = ({E\0}, {IC%C x \0}) = C„ G/G (k) = {{E\0}, 
{Cg|0}, {C 4 |0}, {/|0}, {/C 6 2 |0}, {/C 4 |0}). In P6mm, G (k) = ({E\0}, {Cf |0}, {IC 6 C x \0}, 
{IClCM) = C 2 v, G/G (k) = ({E\0}, {C 6 \0}, {C||0». In P3ml, G (k) = ({E\0}, 
{IClC x \0}) = C„ G/G (k) = ({E\0}, {Cg|0}, {C 4 |0}). 
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Table 7: Optical selection rules at S point. 
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Table 8: Optical selection rules at M point. 



V.F K point 

K = (4vr/3a, 0). In R3m, G (k) = ({E\0}, {C x \0}) = C 2 , G/G (k) = ({E\0}, {C||0}, 
{C 4 |0}, {/|0}, {/C||0}, {/C 4 |0}). In PQmm, G (k) = {{E\0}, {C||0}, {C 4 |0}, {IC'C x \0}, 
{/CfCyo}, {IC%C x \0}) = Csv, G/G (k) = ({E\0}, {C^O}). In P3ml, G (&) = ({^|0}, 
{C 6 2 |0}, {C 4 |0}) = C 3 , G/G (k) = ({E\0}, {IC x \0}). 

For K point of P3ml, K 2 and X 3 are connected by the time reversal operation and 
any physical representation must be decomposed into equal numbers of K2 and K3 rep- 
resentations, as shown in Table 4. Zone center representations are obtained by doing 
the direct products of Ki®Ki, Ki®(K 2 +K 3 ) and (K 2 +K 3 )(g)(K2+K 3 ). 

V.G Z point 

Z = (2£,87r/3a-2£). In R3m, G (k) = ({E\0}) = C u G/G (k) = ({E\0}, {C||0}, 
{C 4 |0}, {C x \0}, {cfcgo}, {ClC x \0}, {I\0}, {ICi\Q}, {/C 4 |0}, {IC X \Q}, {ic%c x \oi {icic x \o}). 
In PQmm, G (k) = ({E\0}, {IC%C x \0}) = C s , G/G (k) = ({E\0}, {C 6 |0}, {C||0}, {Cf|0}, 
{C 4 |0}, {C||0}). In P3ml, G Q (k) = ({E\0}) = C u G/G (k) = ({E\0}, {Cf |0}, {C 4 |0}, 
{IC x \0}, {IClCM, {ICtC x \0}). 



V.H R point 

R = (£, Q is the arbitrary point in the Brillouin zone without any symmetry. In 
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K x K 2 
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rir 3 r 4 r 6 r 2 r 3 r5r 6 
rir 3 r 4 r 6 


rir 2 r 3 
rir 2 r 3 


rir 2 r 3 
rir 2 r 3 


rir 2 r 3 
rir 2 r 3 


PQmm K2 
K 3 




r 4 r 3 


r 2 r 4 
rir 3 


r 5 r 6 
r 5 r 6 
rir 2 r 3 r 4 r 5 r 6 



Table 9: Optical selection rules at K point. 
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Rim z x 


rir 2 r 3 r 4 r 5 r 6 


Tir 2 r 3 Tir 2 r 3 


PQmm ^} 
Z2 




rir 2 r 3 r 4 r 5 r 6 rir 2 r 3 r 4 r 5 r 6 
Tir 2 r 3 r 4 r5r6 



Table 10: Optical selection rules at Z point. 



R3m, Go(fc) = ({E\0}) = &, G/G (k) = ({E\0}, {Cf|0}, {C 6 4 |0}, {C x \0}, {CfC x |0}, 
{ClC x \0}, UIO}, {IC$\0}, UC 6 4 |0}, {IC x \0}, {IC%C x \0}, UCeCxlO}). In PQmm, G (k) 
= ({E\0}) = d, G/G (k) = ({E\0}, {C 6 |0}, {C||0}, {Cf|0}, {C 6 4 |0}, {C||0},{/C x |0}, 
{IClC x \0}, {IC%C X \0}, {IClC x \0}, {IClC x \0}, {IClC x \0}). In P3ml, G (k) = {{E\0}) 
= d, G/G (k) = ({E\0}, {Cl\0}, {Cl\0}, {IC x \0}, {IClC x \0}, {JC|C x |0}). 

While doing the direct products for R3m, PQmm and P3ml at arbitrary point 
R, a general rule about zone center representations in the direct product is discov- 
ered: the direct product of any physical representation at arbitrary point R of the 
Brillouin zone with itself constains zone center representations, the number of ap- 
pearence of the zone center representations equals the dimensionality of the represen- 
tation. For example, without calculation, at arbitrary point R in D\ h , ~kR\ (x) *i? 4 = 

r i" + r 2 + r 3 + r 4 + 2F 5 + r r + r 2 + r 3 + r 4 + 2F 5 > P lus man y terms awa y from 

zone center. The proof is given elsewhere. 

Tables 5-11 cover the entire Brillouin zone of the surface space group PQmm. In 







R3m 


PQmm 






Ri 


Ri 


R3m 


Ri 


Tii^rs^rsre 


rir 2 r 3 


PQmm 


Ri 




rir2r 3 r 4 r 5 r 6 



Table 11: Optical selection rules at R point. 
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tables 5-11, a representation in the direct products is only written down once even if it 
appears many times because one is more concerned about whether or not a transition can 
happen instead of its number of appearances in the direct product. Careful inspection 
of tables 5-11 shows that for optical transitions between bulk state and surface state, 
polarization effects only exist at T, A and M points, where light polarized in different 
directions participates in different transitions. This is in contrast with S, K, Z and R 
points where x, y, z is contained in all direct products therefore light polarized in different 
directions participate in all transitions. The coupling strength is, of course, polarization 
dependent. 

To demonstrate the use of the tables, consider M point (Table 8). p x , p y , p z atomic 
orbits transform as M3, M4, M 4 of space group P6mm and M2, M2, M\ of space group 
R3m. While photons polarized in the (x, y) direction (T3) participate in all bulk to sur- 
face transitions, z direction polarized photons (Ti) only participate in transitions from 
bulk Mi to surface Mi, M4 and from bulk M2 to surface M2, M3. 



VI. Application to photoemission process 

Electron spin polarization (ESP) of photoelectrons of TIs will now be calculated 
to demonstrate the use of the tables. ESP measuments have the advantage that it 
depends entirely and only on the symmetry properties of the wave functions involved 
and is independent of the form of the crystal potential. ESP of photoelectrons has been 
calculated for GaAs and the results agree well with experiments [29] . Instead of the total 
photoelectron intensity, ESP measures the degree of spin polarization, which is defined 
as 

P n-n 
n +1 r 

where I \ (I [) denotes the intensity of spin up (down) electrons. ESP at T point is 
calculated for surface to bulk and bulk to surface transitions. Such transitions are the 
first step of the photoemission process in the "three step model" [30]. 

Taking into account of electron spin, the electronic wave functions are described by 
double groups. In point group D^, the double group representations are r 4 +, T5+ and 
Tq+. In the presence of the time reversal symmetry 6, and Tq + stick together and 
T5- and Tq- stick together. In point group Cq v , the double group representations are 
Tj, Tg and Tg. In point group C^ v , the double group representations are r 4 , Ts and 
Tq, where 6 forces Ts and Fq together. The restrictions of double group representations 
from R3m and PQmm to P3ml at T point are: 

R3m — > -P3ml PQmm — * P3ml 

r 4 + — > r 4 r 7 — > r 4 
r 5+ ©r 6+ ^ r 5 ©r 6 r 8 - r 4 
r 5 -©r6- — r 5 ©r 6 r 9 — r 5 ©r 6 

In ESP calculations double group representations are written down in the form of r], 
which means that the wave function transforms as Ti double group and the spatial part 
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Table 12: Electron spin polarization of photoelectrons excited from bulk band to surface 
band, with right circularly polarized light. Bulk electronic states are on the left and 
surface electronic states are at the top. 
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Table 13: Electron spin polarization of photoelectrons excited from surface band to bulk 
band, with right circularly polarized light. Surface electronic states are on the left and 
bulk electronic states are at the top. 



belongs to Tj single group representation. The results of ESP calculations are tabulated 
in table 12 for bulk to surface transitions and in table 13 for surface to bulk transitions. 
The results are calculated for right circularly polarized light (X — iY) at the center of 
the Brillouin zone. It is known that hybridization effect may lower the magnitude of 
ESP values [31]. 

VII. CONCLUSIONS 

Despite all the possible applications of our analysis, the use of P6mm wall paper 
group must be justified. Surface reconstructions are important for crystal surfaces. It 
has been well studied on the Si (111) surface and several reconstruction patterns were 
found [32, 33]. To achieve lower energy, reconstruction tends to reduce the symmetry of 
the surfaces. The atoms in V 2 VI 3 are stacked in the order of A, B, C, A, ■ ■ -. There 
is a mismatch between A, B and C sites. Seen from the surface layer, the second and 
the third layers (see Fig. 16) have only three fold symmetry. Interactions between the 
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surface layer and inner layers will be three fold like. As a result, the surface structure 
of TIs could lose the Cq v symmetry. However, to our knownledge, no surface recon- 
structions was reported in these systems [34, 35] . The perfect six fold symmetry of the 
Fermi surface is clearly demonstrated in ARPES [8, 11] and STM [36, 37, 38] experi- 
ments. Of perticular interest is reference [8], where both the six fold symmetry of surface 
states and the three fold symmetry of bulk states are resolved simultaneously. These 
observations indicate that surface structure of TIs indeed belongs to P6mm (which has 
been previously noticed by Zhu et al. [39]) and surface electronic wave functions can 
be characterized in P6mm. This paper is not intended to discuss the stability of the 
surface states (that are topologically protected against non-magnetic impurities), but 
rather only focus on their symmetry properties which govern selections rules. 

The actual degree of distortion of surface state wave functions in the z direction due 
to interactions with inner layers requires numerical calculations and is beyond the scope 
of this paper. Wave function distortion in the z direction, however, does not invalidate 
group theoretical calculations because z direction distorted wave functions are still basis 
functions of wallpaper groups. Distortions of wave functions only change the magnitude 
of coupling constants between surface states and bulk states. 

In summary, we performed group theoretical study on topological insulators over the 
entire Brillouin zone. A new method is used to handle transitions between bulk states 
and surface states. It is shown that polarization effects only exist along the TM line. 
The results of electon spin polarization calculations of photoelectrons at Brillouin zone 
center are tabulated. Our work will be helpful for future optical studies on topological 
insulators. It's our next objective to explain the existing optical experiments within the 
framework of present group theoretical analysis, perticular ly for transitions between bulk 
and surface. Many authors may be considering the problem only along certain Brillouin 
zone directions such as TK or TM while the analysis in the entire Brillouin zone is more 
appropriate due to spin momentum locking: the electrons are not in a simple spin up or 
spin down state. 
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